Forecasting Contests
A number of forecasting contests have been conducted in the past. Such contests range from various wagering events, such as Super Bowl pools, to various financial forecasting contests. Typically, such conventional contests seek to identify the best predictor for the outcome of a single event. For example, the website at www.investorsforecast.com as of September, 1999 allowed participants to predict where the Dow Jones Industrial Average (DJIA) will be and what the prices of certain stocks will be at the end of next week. The person submitting the most accurate prediction for the DJIA and the person submitting the most accurate prediction for an individual stock are each given a fixed monetary award, such as $300. Other contests in the financial arena typically allow participants to invest an imaginary amount of money, with the winner being the person whose portfolio is the largest at the end of the contest. One example of such a contest was included in the website at www.fantasystockmarket.com as of September, 1999.
However, the present inventors have discovered that such conventional contests are inadequate in the following respects. First, the rankings generated by such contests typically do not provide useful information for truly identifying the best forecasters. This is a particularly significant shortcoming with respect to financial and economic forecasting, in which it is very useful for third parties to have that information. In addition, these conventional contests often reward short-term or single-event thinking, and such qualities may not be the most desirable in many cases. Finally, partly because of such short-term and single-event thinking, partly because of the specific events for which predictions are solicited in such conventional contests, and partly because of the manner in which such conventional contests are typically structured, the utility of the data produced by such conventional contests for purposes such as combination forecasting often is sub-optimal.
In the financial and economic arenas, the result is that traditionally there has been insufficient data upon which investors could rely in order to select investment advisors. As a result, many investors are left to select advisors based largely on arbitrary criteria or, in the best case, to rely on recommendations from friends. At the same time, many actual and potential investment advisors who are very capable at reading the market conventionally have had very little opportunity to demonstrate their expertise to the public, and thereby attract new clients. Similar concerns exist for other financial and economic experts who wish to demonstrate their expertise or the validity of their prediction techniques.
What is needed, therefore, is a contest in which the rankings and/or rewards are tied more closely to the forecasting characteristics that are most desirable and that yields a large database of information which can serve as the basis for comparing the predictions of different forecasters. It is also desirable that the contest provide data that are statistically significant and can provide the basis for a wide variety of combination forecasts and other statistical analyses as well as being highly useful for marketing purposes.
Prediction Input
In conventional forecasting contests, participants typically submit their predictions by writing, typing or speaking their predictions. Most frequently, such predictions consist of a numerical estimate of what the value of the predicted variable will be at a specified point in time. Thus, for instance, in the www.investorsforecast.com website contest mentioned above, participants typed in the values of their estimates and then submit those estimates by clicking a button on the website.
While such prediction submission techniques are adequate for their intended purpose, they suffer from many shortcomings. The following examples of such shortcomings have been identified by the present inventors.
First, such conventional prediction submission techniques frequently are not very intuitive from the participant's point of view. In particular, they often require the participants to digest a significant amount of information in order to translate their rough feelings about the way the prediction variable is likely to move into a hard number. This is a significant disadvantage for those participants who are very intuitive oriented. Moreover, to the extent such persons are prone to errors in processing such data when converting their rough perceptions into a hard number, their submitted predictions may vary from what they actually believe about the subject variable.
Second, having to enter numerical estimates for each prediction variable can be cumbersome and time-consuming. This may have the effect of limiting the number of variables for which participants are willing to submit predictions.
While other prediction submission techniques have been utilized, they typically have had very limited applicability. For example, the website at www.cyberskipper.com as of September, 1999 permitted participants to compete in predicting certain sports-related events. One of the prediction submission techniques utilized by this site is to display a grid of possible events. The participants can then click on a cell within the grid to designate their prediction that a particular event will occur. Thus, a different grid is displayed for each baseball game, with each row of the grid corresponding to a different baseball player and each column corresponding to a different event (e.g., “runs”, “hits”, home run”). If a participant believes that a certain player will get a home run in a game, he simply clicks on the appropriate cell to enter that prediction. As can be readily appreciated, this technique generally is limited to predicting binary events (i.e., will/will-not occur). In many cases, this deficiency will limit the applicability of such techniques to collection of very coarse predictions.
What is needed, therefore, is a more efficient and intuitive way to enter or submit prediction data that is applicable across a wide range of prediction events and that can permit participants to submit predictions with more specificity than has been available with conventional techniques.
Provision of On-Line Resources
Use of the Internet has become more and more common over the past few years. Similarly, the number of websites on the Internet has grown exponentially and is expected to continue to grow at a fast pace. As a result, the amount of information available on the Internet can be staggering. However, there is often little done to insure that the information provided to end users is the most relevant to those users. A typical website might contain advertising, as well as a certain amount of content. Both types of information are typically controlled exclusively by the owner of the website, possibly based loosely on some indications as to what visitors would like to see, or based on what advertisers might believe will be most effective. However, the present inventors question how good such strategies are at actually providing website visitors with the information that they actually want and, in any event, have concluded that the effectiveness of such conventional strategies must necessarily vary based on the website owner's individual skill in gauging his audience's desires.
Accordingly, the present inventors have discovered that what is needed is a more systematic technique for providing appropriate resources to users over an electronic network, such as the Internet, that more accurately reflects the users' desires.
Financial and Economic Forecasting
The American economy is made up of the simultaneous activities of hundreds of millions of participants, simultaneously buying and selling goods and services in the competitive economy. Probably the most famous market is the Stock Market for the buying and selling of corporate ownership. Each business day, millions of shares of stock are bought and sold at competitive prices. Prices set by the competitive market change as people obtain different information regarding the availability and demand for goods, services, and financial assets. No individual knows all the market conditions in advance of trying to buy or sell. Knowing what prices will be in the future could allow market participants to change the amounts at which they would otherwise transact (e.g., if prices are expected to increase in the near future, knowledgeable sellers might withhold inventory from the market place).
Almost as long as there have been measurements of economic data, people have attempted to formulate forecasts of prices and economic activity by using a variety of techniques. During the past fifty years, several distinct methodologies for producing economic forecasts have been explored. Some of the most important include large-scale econometric systems, time series methods, computationally intensive techniques, opinion polling, and combination methods.
Economists, mathematicians, and forecasters have spent over a century attempting to specify increasingly complex mathematical and statistical models, which, some believe, could allow accurate forecasting to take place. Beginning with economic and behavioral theory, mathematical equations representing the interactions of different variables with each other are hypothesized. Then, using a sophisticated set of econometric model identification techniques, specific numerical values for the equations' parameters are calculated based on historical relationships and observed data. Examples of these models have included the DRI Model, the Wharton Model, and the UCLA Forecasting Project model. Such large multiple equation mathematical forecasting models of the economy are ever increasingly complex, modeling ever-finer levels of economic detail, but their very complexity often makes them inaccurate as forecasting tools.
Some of these models can be used with fair accuracy to provide “what if” simulations for the economy, simulations beginning from a specific initial set of economic measurements and then computing the likely economic impact from various policy changes (e.g. tax cuts, military spending). However, to the extent that the starting values are not precisely measured, or that there are even ever-so-slight errors in the mathematical equations, the resulting forecasts can display extraordinary deviation from the values that eventually are observed in the economy. These problems are made worse if, for any reason, historical economic data were generated by a different set of relationships than are now found in the economy. In this regard, one wag observed that these models are so accurate, economists have successfully predicted 14 of the last 3 recessions. Even so, these large-scale economic forecasting models remain the “gold standard” for economic forecasting, and millions of dollars are spent each year to purchase forecasts from such systems.
Approximately thirty years ago, a group of econometricians, predominantly of British origin, began to develop alternative economic prediction methods. Foremost, single equation models using “time series” techniques popular in engineering applications were found to out-predict the large multiple equation economic models. The development of straightforward computer programs implementing these forecasting techniques allowed for the rapid development of these single equation forecasting models. Numerous economic variables were found to be reasonably predictable using such techniques. These techniques have continued to advance with the development of more complicated techniques (known by acronyms such as “ARCH” and “GARCH”). However, these forecasting techniques are viewed with some suspicion by many economists and forecasters because they lead to models developed using empirical criteria, not models specified as the logical result of economic theory. Even so, single equation forecasting methods are among the most valuable tools used by technical and quantitative market analysts, and are widely applied by Wall Street “Rocket Scientists” and many practicing business forecasters.
Another set of “Rocket Science” tools has become popular during the 1990s, the “computationally intensive” forecasting tools. Using massive computerized databases, mathematical search algorithms are employed to find “black boxes” for forecasting. Such techniques include “neural networks”, large systems of empirically based equations with parameters that evolve over time. Neural networks appear to have been used, for example, in creating the forecasts produced by www.forecasts.org as of September, 1999. Ideally, neural networks learn from their mistakes and self correct. Although neural networks are the foundation of numerous automated trading and arbitrage systems on Wall Street, in practice they sometimes “learn” too slowly and converge on very localized forecasting rules, which do not generalize well.
Still being developed, but of great interest are the computationally intensive statistical pattern matching procedures. Just as the weather service locates historical weather patterns in their database that look like current weather patterns, and then base long term predictions on what the historical “next week's weather” turned out to be, some forecasters are attempting to match past patterns of economic and stock market data to current conditions to make long term predictions. These forecasters are sometimes referred to as the “Rocket Science Technical Forecasters”. However, these techniques are in their infancy and because of sparse historical data may never be of more than limited use in most economic forecasting applications.
In addition, public opinion polls and surveys have been used to forecast “consumer sentiment” measures and to gather data on peoples' consumption patterns. To some extent mirroring the data collection methods used by the government to estimate its official economic measures, these have demonstrated some ability to provide accurate forecasts of what upcoming government statistical releases will say. For instance, the University of Michigan Center for Social Research is identified with its surveyed Index of Consumer Sentiment. Other major public opinion polls also routinely include questions regarding economic conditions.
The final category of forecasts, so-called “consensus forecasts”, is similar to opinion-poll surveys but with a key difference. In public opinion polls, random populations are sampled. In creating a consensus forecast, polls and surveys of economic and financial forecasters (and, sometimes, published forecasts) are conducted. Typically, the median value across participants is the consensus forecast. These surveys have proven to be quite good, generally outperforming over time the individual forecasters who are included in the panel underlying the consensus forecast. Consensus forecasts are regularly conducted for corporate earnings, money supply and interest rates, and key macroeconomic variables. For example, both IBES and First Call survey stock analysts to identify expected corporate earnings. MMS surveys bank economists to estimate the money supply figures on the upcoming Federal Reserve H-6 reports. Blue Chip Economic Indicators was perhaps the first service providing median and average forecasts from a group of forecasters for general economic variables (see www.bluechippubs.com as of September, 1999). The National Association of Business Economists Forecast Survey provides at least quarterly reports on what its membership anticipates for certain general economic variables. The Federal Reserve conducts similar surveys of about 30 economic forecasters with results published regularly in the financial press.
Consensus forecasts are an example of a broader, but relatively infrequently applied category of “combination forecasts”. Combination forecasts are forecasts created from a group of underlying forecasts. Approximately twenty-five years ago, combining forecasts was an active area of econometric research and many theoretical problems were solved, including sophisticated mathematical procedures for determining optimally changing weights for the combinations. Although the consensus forecast median is a combination forecast, median forecasts usually are not the best combination forecasts, given the available data. However, they are “pretty good” combination forecasts, and can be easily calculated.
The consensus forecasts require no historical information about either predictions or accuracy. More sophisticated forecast combinations require a historical track record for each forecast to be included in the combination. Once this track record is available, the forecasts can be analyzed into optimal combinations much like investments are combined into an optimal portfolio.
While consensus forecasting is alive and well, it appears that the broader optimal forecast combination literature has been abandoned or forgotten except, perhaps, in a few academic strongholds. This is not surprising. At the time these theoretical combination techniques were being developed, the efficient market hypothesis was in its prime and stock market forecasts were viewed with great suspicion, if they were considered at all, by academics. Economic forecasts were generally produced on a monthly basis at best, and more often on a quarterly basis. Because virtually all computation was still done on cumbersome mainframe systems, often as overnight batch computation jobs, forecasts were expensive to obtain. Even if a large number of forecasts were available, the optimal combinations could have required more computing power than was readily available to users, just as the Markowitz portfolio problems were generally intractable in practice.
Consequently, the lesson that seemed to be learned from the forecasting combination literature is that people get more accurate predictions if they somehow take an average of forecasts. Hence, demand grew for consensus forecasts based on simple surveys of forecasters, but more advanced combinations were not widely used due to cost, data constraints, and computational complexity. Like many technologies, the optimal forecast combination techniques were developed before the infrastructure was available to allow for their effective implementation.
In addition, combination forecasting can be difficult to implement for a large forecasting panel over a significant period of time, largely because the makeup of the forecasting panel varies over time and because the frequency of participation by the various members of the forecasting panel cannot be adequately controlled.
Still further, in certain cases there may be insufficient forecaster participation to permit a combination forecast of sufficient accuracy. Also, even if an accurate combination forecast is generated for a variable, it may be difficult to say with any certainty what was the relative importance of various factors arriving at the forecast.
Thus, what is needed is a more accurate forecasting methodology that overcomes the above shortcomings in the prior art.
Utilization of Banner Ad Click-Through Information
Many conventional websites include banner advertisements which also function as hyperlinks to the advertiser's website. Thus, if a website visitor is sufficiently interested by the advertisement, he can simply click on the advertisement to retrieve the advertiser's webpage and obtain more information about the particular product or service. Use of such banner advertisements can provide advertising revenue for the displaying website and additional exposure for the advertising company.
In order to better target their advertising efforts, such advertisers might keep track of how many visitors to their site resulted from click-throughs for each of the various banner ads they have posted on others' websites. However, the present inventors have discovered that banner ad click-through information can be used in a wide variety of additional applications, such as further increasing the efficiency of advertisers' marketing efforts, predicting certain events, and others.